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Abstract 

Researchers since at least Darwin have debated whether and to 
what extent emotions are universal or culture-dependent. How- 
ever, previous studies have primarily focused on facial expres- 
sions and on a limited set of emotions. Given that emotions 
have a substantial impact on human lives, evidence for cul- 
tural emotional relativity might be derived by applying distri- 
butional semantics techniques to a text corpus of self-reported 
behaviour. Here, we explore this idea by measuring the valence 
and arousal of the twelve most popular emotion keywords ex- 
pressed on the micro-blogging site Twitter. We do this in three 
geographical regions: Europe, Asia and North America. We 
demonstrate that in our sample, the valence and arousal lev- 
els of the same emotion keywords differ significantly with re- 
spect to these geographical regions — Europeans are, or at 
least present themselves as more positive and aroused. North 
Americans are more negative and Asians appear to be more 
positive but less aroused when compared to global valence and 
arousal levels of the same emotion keywords. Our work is the 
first in kind to programatically map large text corpora to a di- 
mensional model of affect. 

Keywords: Semantic Clustering, Emotion Analysis; Twitter; 
Core Affect Model. 

Introduction 

The question as to whether the experience and expression 
of emotions is universal or relative to specific cultures has 
resulted in a wide variety of studies, with theories ranging 
from the universality hypothesis to culture-specific facial ex- 
pressions. Here we present evidence that culture is a nec- 
essary framework for researchers studying variation in emo- 
tions. Independent of the question of biological differences 
in the experience of emotions, it would be unsurprising if cul- 
ture shapes our conscious perception, expression and experi- 
ence of emotions, as has been hypothesised for other cogni- 
tive phenomena ( Hunt & Agnoli 1991 Fuhrmanetal. 201 l| l. 
Here, we use Latent Semantic Clustering on an emotional text 
corpus mined from Twitter to discern how the primary prop- 
erties normally attributed to emotional keywords — valence 
and arousal — differ as the keywords are used in the same lan- 
guage (English) as exploited across different global regions. 

The Conceptualisation of Emotion Qualia 

Emotion qualia refers to the raw feel of an emotion. The 
actual phenomenon of a particular emotion experienced may 
differ according to each person's perception or understand- 
ing of that emotion, with perception being the result of the 
individual's past and hypothesised responses, unique to each 



keywords are constructed from conceptual knowledge about 
the world, emotions themselves may be concepts that humans 
begin learning in infancy and continuously extend and revise 
throughout life ( Lindquist & Barrettl |2008[ ). This repeated 
experience of labelling a combination of core affect and the 
context in which it occurs as an emotion provides training 
in how to recognise and respond to that emotion. In this 
sense, Barrett describes emotions as simulations. This skill 
of conceptualising core affect as an emotion could be a core 
aspect of emotional intelligence, in much the same way as 
conceptual thinking is core to cognitive intelligence. Each 
person learns the label in association with their unique experi- 
ence, thus each person's conceptualisation of their emotional 
spectrum is unique. Cultures, formed of communicating in- 
dividuals, may therefore also be unique if individual expe- 
riences vary somehow systematically. We base our analysis 
on this hypothesis. The reader should bear in mind that we 
are not analysing emotion keywords in particular, rather, we 
are analysing emotion conceptualisations, or what cultures 
understand specific emotion keywords to mean, using Latent 
Semantic Clustering to infer these meanings. 

Core Affect 

Core affect is an emerging paradigm in affective neuro- 
science, and postulates a continuous approach to defining 
emotions ( |Posner et al.| |2005| ). Several core-affect, or cir- 
cumplex models have been proposed (e.g. Watson & Telle- 
[geiil [1985] |Russell| \mO\ [Cacioppo & B"erntson, ,1994) , yet 
all have one thing in common: they represent emotions as a 
single point in a continuous space defined by two (or rarely 
three) dimensions. Different labels have been assigned to 
these two dominant dimensions by various theorists, such as 
pleasure and engagement, however most commonly, valence 
and arousal are chosen. Thus far, there has been no attempt 
to computationally pinpoint emotions or documents within a 
core affect model using 'online' and 'big' data; to date, re- 
search regarding the core affect model has either been the- 



human being. Barrett ( 2006 1 describes the act of conceptual- 
ising core affect, or in other words, why people attach emo- 
tion labels to the experience of emotion qualia. Since emotion 



oretical (e.g. Watson & Tellegen 1985 i, or conducted via a 
limited survey (e.g. Russell] [1980|. 

Core affect is one of two main theories regarding the rep- 
resentation of emotions, the other being the Basic Emotion 
model, however, neither has thus far received unequivocal 
support. Basic emotions could turn out to map to multiple 
subtypes of coherent emotion networks, but this implies we 
need to split basic emotion categories into further subtypes to 
better reflect these emotion networks ( |Hamann|, ,2012, .Bannj 



20121. Here we extend this view and suggest that the core 



affect model enables us to quantify the properties of the basic 
emotions themselves. 

Previous Work 

There is growing evidence that aspects of a person's psychol- 
ogy can be predicted from their language usage. In the 1990s, 
human semantics was shown to be recoverable from linguis- 



tic corpra independent of any further grounding (Lowe 1997 



Bryson 2008^. Recent applications to individual psychol- 



ogy include discovering individual differences in personal- 
ity fPennebaker & King' '1999]l, discovering cultural change 
in moral beliefs (Bilovich & Bryson [2008j , as well as for 
emotion categorization (Fitzpatrick & Logan 201 l[ l. French 
discovered that co-occurrence techniques such as LSA does 



not detect personality from short text samples (French & Gill 



|2007[ ), but do reveal that texts expressing particular emotions 
have a greater semantic similarity to corresponding exemplar 
words (Gill etal., 2008. ). 

A recent study by Jack et al. ( 2012| l found significant ev- 
idence that facial expressions are indeed culture-dependent; 
that is, different cultures represent the same emotions differ- 
ently. However, whether or not this is because they experi- 
ence different emotion qualia is another question. Using lan- 
guage, rather than facial expressions, as an accessor to emo- 
tion will enable a much more detailed and less ambiguous 
analysis, increasing significance by "throwing more data at 



the problem" ( Recchia & Jones 2009 p. 3). 

Currently, there have been few attempts to analyse cul- 
tural differences using language semantics. Language plays 
a key role in how emotions are conceptualised (and thus per- 
ceived); Lindquist states "language can be no more removed 
from emotion, than flour can be removed from an already 



baked cake" ( Lindquist 2009 p. 1). Recently, Bann& Bryson 



( 2012[ l demonstrated how conceptualisations of emotions can 
be inferred by performing Latent Semantic Analysis on a cor- 
pus of self-reported emotional tweets. Their DELSAR algo- 
rithm analysed 21,000 tweets each labelled with an emotion, 
and clustered each document in the corpus to its most similar 
corresponding emotion label using Latent Semantic Cluster- 
ing. Here we use the same algorithm as the basis for our 
analysis. 

Corpus 

Typing emotion keywords into the Internet is increasingly 
becoming a significant technique for individual expression. 
There now exists a rich available source of information about 
emotions on the Internet, because so many people spend time 
expressing how they feel in blogs, forums, social networking 
websites and the like. We use data from the microblogging 
website Twitter to perform large-scale analysis of the lan- 
guage used in thousands of expressions of emotions within 
tweets. Acquiring a significantly larger corpus than |Bann| 



streaming APlMto create a corpus of 5,625,844 tweeta^ col- 
lected between 19th October 2012 and 18th January 2013. 
Each emotion keyword (see selection criteria below) is given 
a five-minute streaming window in turn for the duration of the 
period, ensuring an even temporal distribution of Tweets is 
collected. Table [T] describes our corpus, split by 'cultural' re- 
gion. We use the tweet's timezone as an indication of the cor- 
responding user's geographical location; seeing as it is very 
unlikely that a Twitter user would select a timezone other than 
that which they reside in, it is somewhat safe to assume that 
this reflects the cultural origin of each user. 

Table 1: Distribution of tweets within our corpus. 



Emotion 


Asia 


Europe 


NA 


All 


Angry 


12194 


27070 


61293 


200024 


Ashamed 


1008 


5097 


17107 


46486 


Calm 


5975 


10181 


36681 


102827 


Depressed 


3078 


11615 


43129 


120473 


Excited 


30923 


100792 


292822 


847679 


Happy 


149129 


186709 


730839 


2201874 


Interested 


3527 


9728 


31891 


86763 


Sad 


46351 


83075 


341912 


966165 


Scared 


15435 


42500 


194130 


517715 


Sleepy 


26031 


10787 


120473 


290666 


Stressed 


2587 


8774 


41716 


109295 


Surprised 


3032 


12454 


56332 


135877 



Total 



299270 508782 1968325 5625844 



Region definitions. We only include those timezones that 
have over 5000 tweets within our corpus. The Asia re- 
gion consists of the timezones Kuala Lumpur, Beijing, Sin- 
gapore, Jakarta, Bangkok, Hong Kong, Tokyo; the Eu- 
rope region consists of the timezones London, Amsterdam, 
Athens, Edinburgh, Dublin, Berlin, Paris; the North Ameri- 
can (NA) region consists of the timezones Eastern Time (US 
& Canada), Central Time (US & Canada), Mountain Time 
(US & Canada), Pacific Time (US & Canada). 
Selection of emotions. As opposed to strictly using the ba- 
sic emotions as identified by 'B arm & Bryson| ( |2012 ), we use 
the most popular emotions that are used on Twitter, that is, 
those emotions that have the highest stream rate. Twelve 
emotions were selected that had a high rate and that equally 
divided into positive/negative and engaged/disengaged theo- 
retical categories (see Table|2]|. 

Subcorpus creation. Each subcorpus is created using a limit 
of 1000 documents per emotion for all subcorpora to ensure 
consistency within our results; we chose 1000 as it is the low- 
est value in Tablefl] To mitigate micro-temporal effects, if the 
number of documents for a particular emotion is significantly 
greater than 1000, we use a modulus function to extract 1000 



'https://dev.twitter.com/docs/streaming-apis. 

^Having first removed 34,725 duplicate tweets. Corpus and code 



& Bryson] ( j2012) , we use the Gardenhose level of Twitter's is available to download at www.aeir.co.uk/code. 



documents equally spaced across the subcorpus — for exam- 
ple, if a particular emotion in a particular subcorpus has 6000 
documents, we take one document every six documents. We 
also create six control subcorpora so to compare our region- 
specific results with a baseline. We use the same modulus 
function to extract 1000 equally spaced tweets, but without 
any timezone clause, selecting six random starting points. 

Proposed Method 

We use DELSAR ( jBann & Brysonl|20T2l see Algorithm[T]l to 
generate the clustering matrix for each subcorpus — the three 
regions Asia, Europe and NA, and six random controls. 

Algorithm 1 DELSAR 

Require: Corpus C and Keyword Set K, where each docu- 
ment in C is mapped to one emotion keyword, emotion, in 
K (through corpus generation) 

Generate cosine document similarity matrix of LSC(C, K) 
(document x document similarity matrix) 
for eacli emotion e K do 

for each document that has emotion emotion do 
delete emotion within the document 
Find the closest document nearest where nearest ^ 
document 

Increment the count for the emotion that nearest is 
labelled as in emotionjvector 
end for each 
return emotion-vector 
end for each 



Table 2: Valence and arousal categorisation of the twelve 
emotion keywords analysed. 



For each subcorpus, DELSAR uses LSA (jLandauer&Du- 
|mais| [TWtI ) to create a document-document matrix of cosine 
similarities {Similarity Matrix), in which similar documents 
are closer to one (i.e. the cosine of the angle between their 
vectors). It creates a clustering matrix that represents the cor- 
pus as an emotion-emotion matrix, describing how each emo- 
tion is similar to each other emotion. 

All analysis was performed on a 64-bit Intel Core i5 CPU 
2x2.67GHz with 4GB RAM using t he GENSIM framework 
for Python (Rehufek & Sojka 2010 1 to create LSA spaces. 



For all tasks, we use a dimension of 36 and use Log-Entropy 
normalisation as our Association Function, found to generate 
optimal results (Nakov et al. 200 l| l and recommended for 
LSA dLandauer & Dumais||1997| l 



Valence and Arousal 

Here we take valance to mean the theoretical positive or neg- 
ative attribution of an emotion keyword, and similarly arousal 
to mean the implied level of engagement. We should use the 
keywords theoretical valence and theoretical arousal as we 
are measuring emotion keywords relative to their generally 
accepted categorisation, although there does seem to be con- 
sistency in these categorisations between theorists. Table [2] 
shows the theoretical definitions of our keywords, accumu- 
lated using several circumplex models of affect (Watson &' 
[Tellegen, 1 1985] [Russell, jl 9801 ICacioppo & Berntson,,1994| ). 



Emotion 


Valence 


Arousal 


Angry 


Negative 


Engaged 


Ashamed 


Negative 


Disengaged 


Calm 


Positive 


Disengaged 


Depressed 


Negative 


Disengaged 


Excited 


Positive 


Engaged 


Happy 


Positive 


Disengaged 


Interested 


Positive 


Engaged 


Sad 


Negative 


Disengaged 


Scared 


Negative 


Engaged 


Sleepy 


Positive 


Disengaged 


Stressed 


Negative 


Engaged 


Surprised 


Positive 


Engaged 



We calculate the valence and arousal levels of each emo- 
tion for each subcorpus as follows. First, we run DELSAR 
on the subcorpus to generate clustering vectors for each emo- 
tion. Each emotion's valence is then calculated as the num- 
ber of positive elements within its vector, as defined in Ta- 
ble l2] divided by the total number of documents across all 
elements (which will always be 1000), or in other words, the 
percentage of positive elements within its vector. Similarly, 
each emotion's arousal is calculated as the percentage of en- 
gaged elements within its vector, again as defined in Table l2] 
We then normalise each valence and arousal value by tak- 
ing away the average valence and arousal value, respectively, 
for all subcorpora analysed — Asia, Europe and NA regions 
and the six control subcorpora. This ensures relativity of the 
resulting circumplex model between these analysed groups; 
these groups can now be compared to one another to estab- 
lish similarities and differences between them. 

Results 

Figure [T] shows a plot of our circumplex of selected subcor- 
pora. We can see that some emotions are more tightly packed 
than others, and interestingly, that low-valence-high-arousal 
and high-valence-low-arousal emotions are much more uni- 
versally similar when compared to the other two quadrants 
of the circumplex. In order to visualise each separate region 
more clearly we illustrate the aggregate theoretical positivity 
and engagement for each subcorpus, shown in Figure [2] This 
clearly illustrates that our three regions do indeed have dif- 
ferent conceptualisations of the same emotion keyword; we 
see that the region Europe is a much more positive and en- 
gaged culture; in other words, Europeans find the same emo- 
tion keywords to be more positive and engaging when com- 
pared to other cultures and indeed our control samples. Also, 
we discover that Asians find the same emotion keywords to 
be somewhat more positive, and North Americans somewhat 
more negative, with negligible arousal differences. 

In order to analyse how tightly packed our emotion clus- 
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Figure 1: Circumplex of three regions and six controls. 



ters are in Figure [T] we conducted K-Means cluster analysis 
to determine the centroids for each emotion, calculating the 
distances of each emotion to its centroid. We plot our cen- 
troids, shown in Figurel3] resulting in a circumplex that could 
be thought of as a universal emotion circumplex, illustrating 
what people think emotions to be, relative to each other emo- 
tion. We can see that the emotions scared, depressed and sad 
have a very similar valence, yet varying arousal levels; so too 
do the emotions sleepy and sad. We can also see, albeit less 
definitively, that the emotions stressed and surprised have a 
similar arousal level, but opposite valence; so too do the emo- 
tions sad and sleepy. 

In order to identify which emotions have the most and least 
similar conceptualisations across cultures, we calculate the 
distance of each emotion to its respective centroid for each 
region, and calculate the sum of these distances for each emo- 
tion across all subcorpora, shown in Table|3] We discover that 



the emotions sad and stressed have the most similar concep- 
tualisations across all cultures; in other words, people under- 
stand these two emotions to mean the same thing independent 
of culture. Similarly, we find that the emotions surprised and 
depressed have the most widely varying conceptualisations 
across cultures; in other words, different cultures have very 
different valence and arousal attributions towards these two 
emotions. Note that we do not include the emotion ashamed 
in the top two due to a strange anomaly in control group 6 
which skews an otherwise relatively tight cluster. 

Discussion 

We would expect that the control groups would be tightly 
clustered around the centre of the circumplex in Figure l2] 
and for the most part, they are. The exceptions are control 
groups one and four, possibly due to the fact the the corpus is 
skewed in favour of tweets originating from NA (see Table [T]i; 
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Figure 2: Aggregate theoretical positivity and engagement for 
each subcorpus. 



Table 3: Sum of subcorpus distances to respective centroids. 



Emotion 


Distance 


Emotion 


Distance 


Sad 


17.94 


Scared 


23.27 


Stressed 


19.66 


Happy 


26.10 


Calm 


20.86 


Excited 


27.00 


Interested 


22.72 


Depressed 


29.56 


Angry 


23.18 


Surprised 


32.89 


Sleepy 


23.19 


Ashamed 


40.70 



this is somewhat verified by their closeness to the NA sub- 
corpus. Other than these anomalous subcorpora, the circum- 
plex does illustrate how different cultures significantly con- 
ceptualise emotions differently, in keywords of valence and 
arousal. Interestingly, there are certain emotions in certain re- 
gions that stick out of our analysis. One example is the emo- 
tion depression; Asians find this emotion much more nega- 
tive than all other cultures and control groups. This could be 



due to cultural differences such as coping strategies (Aldwin 



& Greenberger 1987| l. Another example concerns the emo- 
tions happy and calm; Europeans and Asians find these emo- 
tions much more positive than North Americans and all con- 
trol groups. Another suggests that Asians find interest a very 
positive and aroused emotion, compared to North Americans 
who conceptualise the same emotions, relatively, as negative 
and disengaged. 

Limitations 

We document several limitations of our approach. Firstly, our 
database may still contain duplicate tweets, as some users du- 
plicate tweets by appending, for example, a number at the 
end, making them unique from one another. Second, our 



Figure 3: Centroid emotion circumplex. 

modulus function does not take an even sample for our con- 
trol groups at the country level, so they may be skewed in 
favour of countries with a higher frequency of documents 
within the database (our corpus on the whole is in fact skewed 
in favour of NA). Thirdly, we assume that the emotion key- 
words we have selected are in fact emotion qualia as opposed 
to adjectives. Fourth, our corpus is essentially a snapshot in 
time and may reflect, for example, the political or economic 
climate at the time, or skew due to global events such as the 
US election. Finally, our corpus consists entirely of English 
tweets, which skews our results in favour of Western cultures; 
our Asia, and to some extent, Europe subcorpora may not be 
entirely representative of their respective cultures as we disre- 
gard all native languages other than English. In addition, the 
subpopulations of those regions who choose to use Twitter, 
and do so in English, may be a biased sample. 

Conclusions 

Emotions are being increasingly expressed online, and being 
able to understand these emotions is rapidly becoming a 
concern of AI and Cognitive Science. By mapping culture- 
specific emotion circumplexes, we hope to be better able to 
understand culture-specific perceptions or even experience 
of emotions. From the work presented here we can conclude 
the following: 

Emotional semantics depends on culture. The same emo- 
tion keyword in one culture may describe different valence 
and arousal properties in another. This seems to be more true 
of some keywords than others, and could be critical where, 
for example, a significantly differing conceptualisation of the 
emotion depression would require a different understanding 
and response. 

Emotions vary by geographic region. Europeans are more 
likely to express positiveness and engagement. Asians are 



also more positive than North Americans, both relative to 
each other and to the control subcorpora. Note that this 
may reflect cultural differences in the public expression of 
emotion rather than its actual qualia — our method cannot 
disambiguate these. 

Some emotions do seem to be conceptualised universally. 
The emotion keywords sad and stressed have the same 
conceptualisation across cultures, whereas cultures have 
the most disagreement regarding the conceptualisation of 
surprised. 

We hope that our research paves the way for a better 
understanding of how language can be used to identify 
specific properties of emotions, and we encourage the reader 
to verify our results by downloading our code and corpus at 



http://www.aeir.co.uk/code 
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